Import¶

In [ ]:
import warnings
from sklearn.exceptions import UndefinedMetricWarning

warnings.filterwarnings("ignore", category=UndefinedMetricWarning)
warnings.filterwarnings("ignore", category=UserWarning)
warnings.filterwarnings("ignore", category=FutureWarning)

import plotly
plotly.offline.init_notebook_mode()
In [ ]:
from file_py.run_log_parser import RunLogParser
from file_py.csv_preprocessing_scaler import CsvPreprocessingScaler

from file_py.plots import Plots

from file_py.utils import MarkdownHelper

from file_py.attack_log_unification import AttackLogUnification
from file_py.stat_severity import StatSeverity
from file_py.attack_pattern_analyzer import AttackPatternAnalyzer
from file_py.signatures_patterns import SignaturePatterns

from file_py.signature_stats_calculator import SignatureStatsCalculator
from file_py.sigma_rule_analysis import SigmaRuleAnalysis

from file_py.plots_single_attack import PlotsSingleAttack

from file_py.correlation_matrix_plots import CorrelationMatrixPlots

from file_py.preprocessing_train_test_split import PreprocessingTrainTestSplit
from file_py.initial_training import InitialTraining
from file_py.hyperparameter_tuning import HyperparameterTuning
from file_py.advanced_models import AdvancedModels
from file_py.deep_learning_model import DeepLearningModel
from file_py.model_evaluator import ModelEvaluator

CARICAMENTO FILE¶

Sostituire il percorso dei file attuali con il percorso dei file di interesse qui:

In [ ]:
# FILE CONTENENTE I LOG
df = CsvPreprocessingScaler.read_csv_file("file_csv/Log_splunk_combined.csv")

# FILE CON LE DATE DI INIZIO E FINE DEGLI ATTACCHI
files = ['file_csv/attackLog_combined.csv']

Preprocessing¶

In [ ]:
df_raw = CsvPreprocessingScaler.RawPreprocessing(df)
df_Le = CsvPreprocessingScaler.LEPreprocessing(df)
df_OH = CsvPreprocessingScaler.OhePreprocessing(df)
In [ ]:
df_std_LE = CsvPreprocessingScaler.stdScaler(CsvPreprocessingScaler.LEPreprocessing(df))
df_std_OH = CsvPreprocessingScaler.stdScaler(CsvPreprocessingScaler.OhePreprocessing(df))

Test¶

In [ ]:
attack_log_path = AttackLogUnification.attack_log_together( files,'file_csv/attackLog_combined.csv')
In [ ]:
result_df_Le = RunLogParser.process_attacks(attack_log_path, CsvPreprocessingScaler.stdScaler(CsvPreprocessingScaler.LEPreprocessing(df)))
result_df_OH = RunLogParser.process_attacks(attack_log_path, CsvPreprocessingScaler.stdScaler(CsvPreprocessingScaler.OhePreprocessing(df)))
result_df_Raw = RunLogParser.process_attacks(attack_log_path, CsvPreprocessingScaler.RawPreprocessing(df))

Graphic Analysis of Attacks¶

In [ ]:
Plots.plot_cake_attack(result_df_Raw)
No description has been provided for this image
In [ ]:
Plots.plot_top_10_signatures(result_df_Raw)
Out[ ]:

Qui si può notare come generalmente le regole scattate più volte sono anche quelle che hanno effettivamente risposto a più attachi e che sono scattate a vuoto più volte.

In [ ]:
Plots.plot_precision_recall(result_df_Raw)

Il primo grafico mostra la precisione di ciascuna regola, cioè la proporzione di attivazioni corrette rispetto al totale delle sue attivazioni.
Una precisione più alta indica che la regola è più accurata nel rilevare veri attacchi.

Il secondo grafico mostra il recall, cioè la proporzione di attacchi reali rilevati dalla regola rispetto al totale degli attacchi reali.
Un recall più alto indica che la regola è più efficace nel rilevare tutti gli attacchi possibili.

In [ ]:
Plots.plot_distributions(result_df_Raw)
No description has been provided for this image
In [ ]:
Plots.plot_value_counts_per_unique(result_df_Raw)
No description has been provided for this image
In [ ]:
variables = MarkdownHelper.create_value_counts_variables(result_df_Raw)
MarkdownHelper.display_value_counts_text(variables)
Grazie a questo grafico invece possiamo giungere ad una serie di conclusioni.

Su 103 regole diverse:

  • quelle scattate in risposta ad ALMENO un attacco reale sono 88. Di queste:

    • 27 si sono attivate più volte per non-attacchi rispetto che per gli attacchi. (regole generiche)
    • 20 si sono attivate lo stesso numero di volte per attacchi e non-attacchi.
    • 41 si sono attivate più volte in risposta ad attacchi rispetto che a non-attacchi (regole specifiche).
  • quelle scattate senza rispondere mai ad attacchi sono 15.

    Si tratta di: ['proc-start-copy-from-volumeshadowcopy-via-cmd.exe', 'proc-start-service-registry-key-deleted-via-reg.exe', 'proc-start-pua-suspicious-activedirectory-enumeration-via-adfind.exe', 'proc-start-suspicious-mshta-child-process', 'proc-start-scheduled-task-executing-encoded-payload-from-registry', 'proc-start-run-powershell-script-from-ads', 'proc-start-file-in-suspicious-location-encoded-to-base64-via-certutil.exe', 'proc-start-network-reconnaissance-activity', 'proc-start-process-memory-dump-via-rdrleakdiag.exe', 'proc-start-renamed-createdump-utility-execution', 'proc-start-potential-persistence-via-logon-scripts-commandline', 'proc-start-suspicious-process-patterns-ntds.dit-exfil', 'proc-start-lolbas-encode-decode', 'proc-start-whoami.exe-execution-anomaly', 'proc-start-lsass-process-reconnaissance-via-findstr.exe']

Analysis of Severity per Attacks¶

In [ ]:
event_df = RunLogParser.create_event_df(attack_log_path, result_df_Raw)

Creazione del df event_df con le nuove colonne severity_max, _min, _mean

Grafici¶

In [ ]:
StatSeverity.plot_stat_severity(event_df)
No description has been provided for this image

In questo grafico vediamo, per ciascun attacco presente nel dataset, quali sono le loro criticità massime, minime e medie.

In [ ]:
analyzer = AttackPatternAnalyzer(event_df)
In [ ]:
# SCEGLIERE UN VALORE PER LA SEVERITY DELLE REGOLE DA CONSIDERARE
severity_value=73

# SCEGLIERE IL NUMERO DI ATTACCHI DA CONSIDERARE PRIMA DELLE REGOLE AVENTI LA SEVERITY SCELTA
num_attacks=10
In [ ]:
analyzer.pattern_before_attack(num_attacks=num_attacks, severity_value=severity_value)
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image

In questi grafici prendiamo in considerazione gli attacchi precedenti a tutti gli attacchi che hanno una certa criticità media e visualizziamo tutti i valori di "RuleAnnotation.mitre_attack.id", "signature", "EventType", "tag", "severity_id" corrispondenti.

Per scegliere quanti attacchi prima di quelli che ci interessano vogliamo considerare basta modificare la variabile "num_attacks" e assegnarle il numero che vogliamo,
mentre per scegliere il valore della criticità media che ci interessa si deve modificare la variabile "severity_value".
Verranno presi in considerazione tutti gli attacchi con criticità media compresa tra 2.5 prima e 2.5 dopo del valore assegnato a "severity_value".

Robustezza regole¶

In [ ]:
signature_stats = SignatureStatsCalculator.create_signature_stats(event_df, result_df_Raw)
signature_stats
Out[ ]:
signature Indice_Diff Media_Differenza_Severity_min Media_Differenza_Severity_mean Media_Differenza_Severity_max N_Max_Sev_Diff_15 N_Attacchi_Non_rilevati
0 proc-start-dumping-of-sensitive-hives-via-reg.exe 0.000421 0.000000 0.068447 0.000000 0 0
1 proc-start-copying-sensitive-files-with-creden... 0.001330 0.000000 0.059737 0.000000 0 0
2 suspicious-volume-shadow-copy-vss_ps.dll-load 0.000882 0.000000 0.005974 0.000000 0 0
3 net-connect-80-443-non-browser 0.054409 -9.537037 -2.637203 0.000000 0 9
4 proc-start-malicious-powershell-commandlets-pr... 0.011164 0.000000 0.325955 0.179856 1 1
... ... ... ... ... ... ... ...
98 proc-start-renamed-createdump-utility-execution 0.000000 0.000000 0.000000 0.000000 0 0
99 proc-start-process-memory-dump-via-rdrleakdiag... 0.000000 0.000000 0.000000 0.000000 0 0
100 proc-start-abused-debug-privilege-by-arbitrary... 0.000052 0.000000 0.000000 0.000000 0 0
101 proc-start-potential-cobaltstrike-process-patt... 0.000034 0.000000 0.000000 0.000000 0 0
102 proc-start-potential-meterpreter/cobaltstrike-... 0.000072 0.000000 0.000000 0.000000 0 0

103 rows × 7 columns

signature_stats è un dataset in cui possiamo vedere per ogni regola se venisse rimossa quali cambiamenti di severity apporterebbe al dataset dei log e se ci dovessero essere degli attacchi che non vengono rilevati.

In [ ]:
analysis = SigmaRuleAnalysis(signature_stats)
analysis.plots_sigma_rule_analysis()
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image

Graphic Analysis of Attacks for Chosen Rule¶

In [ ]:
# SCEGLIERE LA REGOLA CHE SI VUOLE ANALIZZARE
regola_scelta = 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'
In [ ]:
PlotsSingleAttack.analyze_rule_activations(result_df_Raw, regola_scelta)
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image

In questi grafici in base alla regola che si vuole analizzare possiamo visualizzare:

  • la frequenza delle attivazioni delle regole (attacchi e non attacchi) suddivise in intervalli di 5 minuti;
  • gli attacchi e i non-attacchi in base a:
    • RuleAnnotation.mitre_attack.id
    • EventType,
    • severity,
    • tag
    • parent_process_id,
    • process_id
In [ ]:
# SCEGLIERE IL NUMERO DI EVENTI PRECEDENTI ALLA REGOLA CHE SI VOGLIONO ANALIZZARE
eventi_da_considerare = 5
In [ ]:
PlotsSingleAttack.patterns_before_activation(result_df_Raw, regola_scelta, eventi_da_considerare)
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image
No description has been provided for this image

In questi grafici vediamo quali sono rispettivamente le regole, gli attacchi, gli EventType, i tag, i parent_process, i process e le severity degli eventi subito prima delle prime attivazioni della regola scelta.
Il numero di eventi da considerare lo scegliamo affidando alla variabile elementi_da_considerare il numero che vogliamo.

Con "prima attivazione di una regola" si intende quando almeno un elemento delle colonne signature, RuleAnnotation.mitre_attack.id, EventType, tag, severity_id, parent_process_id o process_id (non sono considerate solo le colonne _time e corrisponde_ad_attacco) di un evento differisce da quello precedente.

Patterns¶

In [ ]:
signature_patterns = SignaturePatterns.recognize_signatures_patterns(result_df_Raw)
signature_patterns
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 424
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 389
Pattern: ('proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths'), Frequenza: 30
Pattern: ('proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths'), Frequenza: 28
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 27
Pattern: ('proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths'), Frequenza: 26
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 23
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 23
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 23
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser'), Frequenza: 20
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser'), Frequenza: 20
Pattern: ('net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 17
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser'), Frequenza: 17
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names'), Frequenza: 17
Pattern: ('net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 15
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 13
Pattern: ('proc-start-suspicious-process-parents', 'proc-start-suspicious-process-parents'), Frequenza: 13
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 11
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 11
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 11
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 11
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names'), Frequenza: 11
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names'), Frequenza: 11
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 11
Pattern: ('proc-start-suspicious-process-parents', 'proc-start-suspicious-process-parents', 'proc-start-suspicious-process-parents'), Frequenza: 11
Pattern: ('net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 10
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 10
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 10
Pattern: ('reg-value-write-cert-change', 'reg-value-write-cert-change', 'reg-value-write-cert-change', 'reg-value-write-cert-change'), Frequenza: 10
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 9
Pattern: ('net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 9
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 9
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 9
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 9
Pattern: ('proc-start-suspicious-process-parents', 'proc-start-suspicious-process-parents', 'proc-start-suspicious-process-parents', 'proc-start-suspicious-process-parents'), Frequenza: 9
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes'), Frequenza: 8
Pattern: ('net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 8
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes'), Frequenza: 8
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-80-443-non-browser'), Frequenza: 8
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser'), Frequenza: 7
Pattern: ('net-connect-Windows-processes', 'net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 7
Pattern: ('net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-80-443-non-browser'), Frequenza: 7
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser'), Frequenza: 7
Pattern: ('net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 7
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-80-443-non-browser'), Frequenza: 7
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 7
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 7
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 7
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 7
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 7
Pattern: ('net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 6
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes', 'net-connect-Windows-processes'), Frequenza: 6
Pattern: ('net-connect-Windows-processes', 'net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 6
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-Windows-processes'), Frequenza: 6
Pattern: ('net-connect-Windows-processes', 'net-connect-80-443-non-browser', 'net-connect-Windows-processes', 'net-connect-Windows-processes'), Frequenza: 6
Pattern: ('proc-start-dns-exfiltration-nslookup', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 6
Pattern: ('proc-start-suspicious-program-names', 'proc-start-hacktool-sharpview-execution'), Frequenza: 6
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names', 'proc-start-hacktool-sharpview-execution'), Frequenza: 6
Pattern: ('proc-start-suspicious-program-names', 'proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution'), Frequenza: 6
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names', 'proc-start-hacktool-sharpview-execution'), Frequenza: 6
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names', 'proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution'), Frequenza: 6
Pattern: ('proc-start-suspicious-program-names', 'proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution'), Frequenza: 6
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 5
Pattern: ('net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 5
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes', 'net-connect-Windows-processes'), Frequenza: 5
Pattern: ('net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-80-443-non-browser'), Frequenza: 5
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 5
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 5
Pattern: ('proc-start-suspicious-program-names', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 5
Pattern: ('proc-start-suspicious-program-names', 'proc-start-powershell-download-and-execution-cradles'), Frequenza: 5
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-potentially-suspicious-powershell-child-processes'), Frequenza: 5
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-potentially-suspicious-powershell-child-processes'), Frequenza: 5
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 5
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 5
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-suspicious-target-names'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-suspicious-target-names'), Frequenza: 4
Pattern: ('potential-system-dll-sideloading-from-non-system-locations', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 4
Pattern: ('potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 4
Pattern: ('potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 4
Pattern: ('net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-80-443-non-browser', 'net-connect-Windows-processes'), Frequenza: 4
Pattern: ('proc-start-suspicious-program-names', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 4
Pattern: ('proc-start-suspicious-program-names', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 4
Pattern: ('proc-start-suspicious-program-names', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 4
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-80-443-non-browser', 'net-connect-Windows-processes'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-80-443-non-browser', 'net-connect-Windows-processes'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 4
Pattern: ('proc-start-hacktool-sharpview-execution', 'proc-start-suspicious-program-names'), Frequenza: 4
Pattern: ('proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution', 'proc-start-suspicious-program-names'), Frequenza: 4
Pattern: ('proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution', 'proc-start-suspicious-program-names'), Frequenza: 4
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-Windows-processes'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-Windows-processes'), Frequenza: 4
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-Windows-processes', 'net-connect-80-443-non-browser'), Frequenza: 4
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-Windows-processes', 'net-connect-80-443-non-browser'), Frequenza: 4
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 4
Pattern: ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'), Frequenza: 4
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-suspicious-target-names', 'net-connect-Windows-processes'), Frequenza: 3
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-suspicious-target-names', 'net-connect-Windows-processes'), Frequenza: 3
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-suspicious-target-names', 'net-connect-Windows-processes', 'net-connect-80-443-non-browser'), Frequenza: 3
Pattern: ('net-connect-suspicious-sources', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'net-connect-suspicious-sources', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('net-connect-suspicious-sources', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'net-connect-suspicious-sources', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('net-connect-suspicious-sources', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'potential-system-dll-sideloading-from-non-system-locations'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'potential-system-dll-sideloading-from-non-system-locations'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations'), Frequenza: 3
Pattern: ('potential-system-dll-sideloading-from-non-system-locations', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'potential-system-dll-sideloading-from-non-system-locations'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations'), Frequenza: 3
Pattern: ('potential-system-dll-sideloading-from-non-system-locations', 'potential-system-dll-sideloading-from-non-system-locations', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'net-connect-suspicious-sources', 'net-connect-80-443-non-browser', 'net-connect-80-443-non-browser'), Frequenza: 3
Pattern: ('proc-start-suspicious-double-extension-file-execution', 'proc-start-dir-user-writeable'), Frequenza: 3
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('proc-start-powershell-download-and-execution-cradles', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-80-443-non-browser', 'net-connect-Windows-processes', 'net-connect-suspicious-target-names'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'reg-value-write-cert-change', 'reg-value-write-cert-change'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'reg-value-write-cert-change', 'reg-value-write-cert-change', 'reg-value-write-cert-change'), Frequenza: 3
Pattern: ('proc-start-dir-user-writeable', 'reg-value-write-cert-change'), Frequenza: 3
Pattern: ('reg-value-write-cert-change', 'proc-start-dir-user-writeable'), Frequenza: 3
Pattern: ('proc-start-dir-user-writeable', 'reg-value-write-cert-change', 'reg-value-write-cert-change'), Frequenza: 3
Pattern: ('reg-value-write-cert-change', 'reg-value-write-cert-change', 'proc-start-dir-user-writeable'), Frequenza: 3
Pattern: ('proc-start-dir-user-writeable', 'reg-value-write-cert-change', 'reg-value-write-cert-change', 'reg-value-write-cert-change'), Frequenza: 3
Pattern: ('reg-value-write-cert-change', 'reg-value-write-cert-change', 'reg-value-write-cert-change', 'proc-start-dir-user-writeable'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-mavinject-inject-dll-into-running-process'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-mavinject-inject-dll-into-running-process'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process'), Frequenza: 3
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 3
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 3
Pattern: ('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-suspicious-process-parents'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-suspicious-process-parents'), Frequenza: 3
Pattern: ('proc-start-hacktool-mimikatz-execution', 'proc-start-suspicious-program-names'), Frequenza: 3
Pattern: ('proc-start-hacktool-mimikatz-execution', 'proc-start-hacktool-mimikatz-execution', 'proc-start-suspicious-program-names'), Frequenza: 3
Pattern: ('proc-start-hacktool-mimikatz-execution', 'proc-start-hacktool-mimikatz-execution', 'proc-start-hacktool-mimikatz-execution', 'proc-start-suspicious-program-names'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-Windows-processes'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'net-connect-Windows-processes', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-potentially-suspicious-powershell-child-processes'), Frequenza: 3
Pattern: ('net-connect-80-443-non-browser', 'net-connect-Windows-processes', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'), Frequenza: 3
Pattern: ('proc-start-suspicious-reconnaissance-activity-via-gathernetworkinfo.vbs', 'proc-start-suspicious-reconnaissance-activity-via-gathernetworkinfo.vbs'), Frequenza: 3
Pattern: ('proc-start-potential-lsass-process-dump-via-procdump', 'proc-start-potential-lsass-process-dump-via-procdump'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('proc-start-dns-exfiltration-nslookup', 'net-connect-Windows-processes', 'proc-start-dns-exfiltration-nslookup', 'proc-start-dns-exfiltration-nslookup'), Frequenza: 3
Pattern: ('proc-start-potential-meterpreter/cobaltstrike-activity', 'proc-start-potential-meterpreter/cobaltstrike-activity'), Frequenza: 3

In signature_patterns vediamo le sequenze di 3, 4 o 5 regole in ordine da quella più a quella meno frequente ripetutesi più volte durante i vari attacchi e che non compaiono mai tra le sequenze di falsi attacchi

With specified severity value¶

In [ ]:
result_pattern_inside_attack = analyzer.pattern_inside_attack(severity_value=severity_value)
result_pattern_inside_attack
MITRE ATT&CK IDs:
1-digit repetitions:

2-digits sequences:
  ('T1003.001', 'T1003.001'): 23
  ('T1055.001', 'T1055.001'): 3
  ('T1059', 'T1059.001'): 3
  ('T1003.002', 'T1003.002'): 2
  ('T1070.001', 'T1070.001'): 2
  ('T1003', 'T1036'): 2
  ('T1021.001', 'T1021.001'): 1
  ('T1053.005', 'T1053.005'): 1
  ('T1016', 'T1016'): 1
  ('T1059.001', 'T1059'): 1
  ('T1105', 'T1059.001'): 1
  ('T1070', 'T1070'): 1

3-digits sequences:
  ('T1482', 'T1482', 'T1482'): 21
  ('T1003.001', 'T1003.001', 'T1003.001'): 11
  ('T1059', 'T1003.001', 'T1003.001'): 4
  ('T1049', 'T1049', 'T1049'): 4
  ('T1027', 'T1027', 'T1027'): 3
  ('T1106', 'T1003.001', 'T1003.001'): 3
  ('T1112', 'T1112', 'T1562.001'): 2
  ('T1003.001', 'T1548', 'T1036'): 2
  ('T1003.001', 'T1003.001', 'T1059.001'): 2
  ('T1003.001', 'T1003.001', 'T1482'): 2
  ('T1059', 'T1059.001', 'T1105'): 2
  ('T1059', 'T1059.001', 'T1003.001'): 2
  ('T1003.001', 'T1003.001', 'T1059'): 2
  ('T1059.001', 'T1482', 'T1482'): 2
  ('T1105', 'T1482', 'T1482'): 1
  ('T1003.002', 'T1003.002', 'T1003.003'): 1
  ('T1003.001', 'T1003.001', 'T1543.003'): 1
  ('T1105', 'T1018', 'T1018'): 1
  ('T1059', 'T1059', 'T1482'): 1
  ('T1033', 'T1003.001', 'T1003.001'): 1
  ('T1548', 'T1105', 'T1003.001'): 1
  ('T1036.003', 'T1566.001', 'T1548'): 1
  ('T1059.001', 'T1105', 'T1003.001'): 1
  ('T1105', 'T1105', 'T1003.001'): 1
  ('T1106', 'T1059.001', 'T1482'): 1
  ('T1548', 'T1036', 'T1003.001'): 1
  ('T1055.001', 'T1055.001', 'T1218.013'): 1
  ('T1070', 'T1070', 'T1485'): 1
  ('T1059.001', 'T1059', 'T1059.001'): 1
  ('T1059.001', 'T1059', 'T1482'): 1
  ('T1036', 'T1036', 'T1003.001'): 1
  ('T1003.001', 'T1003.001', 'T1036'): 1
  ('T1003.001', 'T1059.001', 'T1059'): 1
  ('T1105', 'T1105', 'T1105'): 1
  ('T1105', 'T1543.003', 'T1003.001'): 1
  ('T1036.003', 'T1548', 'T1036'): 1
  ('T1036.003', 'T1548', 'T1566.001'): 1
  ('T1548', 'T1588.003', 'T1588.003'): 1
  ('T1059', 'T1059', 'T1059.001'): 1
  ('T1059', 'T1059.001', 'T1059'): 1
  ('T1018', 'T1018', 'T1018'): 1
  ('T1564.004', 'T1133', 'T1133'): 1
  ('T1059.001', 'T1555.004', 'T1218.011'): 1
  ('T1059.001', 'T1059', 'T1105'): 1

SIGNATURES:
1-digit repetitions:

2-digits sequences:
  ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 23
  ('proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process'): 3
  ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'): 3
  ('proc-start-suspicious-eventlog-clear-or-configuration-change', 'proc-start-suspicious-eventlog-clear-or-configuration-change'): 2
  ('proc-start-potential-credential-dumping-attempt-using-new-networkprovider-cli', 'proc-start-process-memory-dump-via-comsvcs.dll'): 2
  ('proc-start-volumeshadowcopy-symlink-creation-via-mklink', 'proc-start-volumeshadowcopy-symlink-creation-via-mklink'): 1
  ('proc-start-potential-tampering-with-rdp-related-registry-keys-via-reg.exe', 'proc-start-potential-tampering-with-rdp-related-registry-keys-via-reg.exe'): 1
  ('proc-start-suspicious-command-patterns-in-scheduled-task-creation', 'proc-start-schtasks-creation-or-modification-with-system-privileges'): 1
  ('proc-start-copying-sensitive-files-with-credential-data', 'proc-start-copying-sensitive-files-with-credential-data'): 1
  ('proc-start-potential-recon-activity-via-nltest.exe', 'proc-start-potential-recon-activity-via-nltest.exe'): 1
  ('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles'): 1
  ('proc-start-suspicious-invoke-webrequest-execution', 'proc-start-potential-data-exfiltration-activity-via-commandline-tools'): 1
  ('proc-start-fsutil-suspicious-invocation', 'proc-start-fsutil-suspicious-invocation'): 1

3-digits sequences:
  ('proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'): 21
  ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 9
  ('proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution', 'proc-start-hacktool-sharpview-execution'): 4
  ('proc-start-base64-encoded-powershell-command-detected', 'proc-start-base64-encoded-powershell-command-detected', 'proc-start-base64-encoded-powershell-command-detected'): 3
  ('proc-start-potential-winapi-calls-via-commandline', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 3
  ('proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths', 'proc-start-reg-add-suspicious-paths'): 2
  ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-dir-user-writeable', 'proc-start-system-file-execution-location-anomaly'): 2
  ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-malicious-powershell-commandlets-processcreation'): 2
  ('proc-start-suspicious-program-names', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 2
  ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'net-connect-suspicious-target-names'): 2
  ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 2
  ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-powershell-download-and-execution-cradles'): 2
  ('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'): 2
  ('net-connect-80-443-non-browser', 'proc-start-malicious-powershell-commandlets-processcreation', 'proc-start-malicious-powershell-commandlets-processcreation'): 1
  ('proc-start-copying-sensitive-files-with-credential-data', 'proc-start-copying-sensitive-files-with-credential-data', 'proc-start-copying-sensitive-files-with-credential-data'): 1
  ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'reg-key-create-service'): 1
  ('net-connect-80-443-non-browser', 'proc-start-pua-adfind-suspicious-execution', 'proc-start-pua-adfind-suspicious-execution'): 1
  ('proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-program-names', 'proc-start-malicious-powershell-commandlets-processcreation'): 1
  ('proc-start-suspicious-whoami.exe-execution', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
  ('proc-start-dir-user-writeable', 'net-connect-80-443-non-browser', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
  ('proc-start-lol-binary-copied-from-system-directory', 'proc-start-suspicious-double-extension-file-execution', 'proc-start-dir-user-writeable'): 1
  ('proc-start-potential-data-exfiltration-activity-via-commandline-tools', 'proc-start-suspicious-invoke-webrequest-execution', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
  ('proc-start-hacktool-mimikatz-execution', 'proc-start-hacktool-mimikatz-execution', 'proc-start-hacktool-mimikatz-execution'): 1
  ('net-connect-80-443-non-browser', 'net-connect-80-443-non-browser', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
  ('proc-start-potential-winapi-calls-via-commandline', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-malicious-powershell-commandlets-processcreation'): 1
  ('proc-start-dir-user-writeable', 'proc-start-suspicious-process-parents', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
  ('proc-start-powershell-download-and-execution-cradles', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
  ('proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process', 'proc-start-mavinject-inject-dll-into-running-process'): 1
  ('proc-start-powershell-download-and-execution-cradles', 'proc-start-hacktool-mimikatz-execution', 'proc-start-hacktool-mimikatz-execution'): 1
  ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-suspicious-powershell-download-and-execute-pattern'): 1
  ('proc-start-fsutil-suspicious-invocation', 'proc-start-fsutil-suspicious-invocation', 'proc-start-fsutil-suspicious-invocation'): 1
  ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-potentially-suspicious-powershell-child-processes'): 1
  ('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-potentially-suspicious-powershell-child-processes'): 1
  ('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-malicious-powershell-commandlets-processcreation'): 1
  ('proc-start-potential-lsass-process-dump-via-procdump', 'proc-start-potential-lsass-process-dump-via-procdump', 'proc-start-potential-lsass-process-dump-via-procdump'): 1
  ('proc-start-lsass-dump-keyword-in-commandline', 'proc-start-lsass-dump-keyword-in-commandline', 'proc-start-process-memory-dump-via-comsvcs.dll'): 1
  ('suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded', 'proc-start-lsass-dump-keyword-in-commandline'): 1
  ('proc-start-lsass-dump-keyword-in-commandline', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles'): 1
  ('net-connect-Windows-processes', 'net-connect-Windows-processes', 'net-connect-Windows-processes'): 1
  ('proc-start-suspicious-invoke-webrequest-execution', 'reg-key-create-service', 'suspicious-unsigned-dbghelp/dbgcore-dll-loaded'): 1
  ('proc-start-lol-binary-copied-from-system-directory', 'proc-start-dir-user-writeable', 'proc-start-system-file-execution-location-anomaly'): 1
  ('proc-start-lol-binary-copied-from-system-directory', 'proc-start-dir-user-writeable', 'proc-start-suspicious-double-extension-file-execution'): 1
  ('proc-start-dir-user-writeable', 'reg-value-write-cert-change', 'reg-value-write-cert-change'): 1
  ('proc-start-suspicious-program-names', 'proc-start-powershell-download-and-execution-cradles', 'proc-start-suspicious-powershell-download-and-execute-pattern'): 1
  ('proc-start-suspicious-program-names', 'proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles'): 1
  ('proc-start-renamed-adfind-execution', 'proc-start-renamed-adfind-execution', 'proc-start-renamed-adfind-execution'): 1
  ('proc-start-lolbas-alternate-data-streams', 'proc-start-suspicious-add-user-to-remote-desktop-users-group', 'proc-start-suspicious-add-user-to-remote-desktop-users-group'): 1
  ('proc-start-potentially-suspicious-powershell-child-processes', 'proc-start-suspicious-key-manager-access', 'proc-start-rundll32-execution-without-dll-file'): 1
  ('proc-start-suspicious-powershell-download-and-execute-pattern', 'proc-start-powershell-download-and-execution-cradles', 'net-connect-suspicious-target-names'): 1

In result_pattern_inside_attack vediamo:

  • '1-digit repetitions' che corrisponde alle ripetizioni di mitre_attack.id e signature in testa agli attacchi con un numero massimo di 3 mitre o signature registrati;
  • '2-digits sequences' che corrisponde alle sequenze di mitre_attack.id e signature in testa agli attacchi con un numero di mitre o signature compreso tra 4 e 5;
  • '3-digits sequences' che corrisponde alle sequenze di mitre_attack.id e signature in testa agli attacchi con un numero di mitre o signature maggiore di 5 (non compreso).

Correlation Matrix¶

In [ ]:
CorrelationMatrixPlots.plot_correlation_matrix(result_df_Le, 'Correlation Matrix (Label Encoding)')
CorrelationMatrixPlots.plot_correlation_matrix_big(result_df_OH, 'Correlation Matrix (OneHot Encoding)')
No description has been provided for this image
No description has been provided for this image

ML¶

OneHot¶

In [ ]:
# Split data
X_train_OH, X_test_OH, y_train_OH, y_test_OH = PreprocessingTrainTestSplit.split_data(result_df_OH, "corrisponde_ad_attacco")

# Initial model training and evaluation
InitialTraining.train_and_evaluate_initial_models(X_train_OH, y_train_OH, X_test_OH, y_test_OH)

# Hyperparameter tuning
best_models_OH = HyperparameterTuning.tune_hyperparameters(X_train_OH, y_train_OH)

# Evaluate best models on test set
evaluator_OH = ModelEvaluator(best_models_OH)
evaluation_results_OH = evaluator_OH.evaluate_models(X_test_OH, y_test_OH)

# Train XGBoost model
AdvancedModels.train_xgboost(X_train_OH, y_train_OH, X_test_OH, y_test_OH)

# Train deep learning model
DeepLearningModel.train_deep_learning_model(X_train_OH, y_train_OH, X_test_OH, y_test_OH)
Decision Tree Classification Report:
              precision    recall  f1-score   support

           0       0.83      0.82      0.82       742
           1       0.92      0.93      0.92      1692

    accuracy                           0.89      2434
   macro avg       0.87      0.87      0.87      2434
weighted avg       0.89      0.89      0.89      2434


AdaBoost Classification Report:
              precision    recall  f1-score   support

           0       0.73      0.38      0.50       742
           1       0.78      0.94      0.85      1692

    accuracy                           0.77      2434
   macro avg       0.75      0.66      0.68      2434
weighted avg       0.76      0.77      0.74      2434


XGBoost Classification Report:
              precision    recall  f1-score   support

           0       0.85      0.76      0.80       742
           1       0.90      0.94      0.92      1692

    accuracy                           0.89      2434
   macro avg       0.88      0.85      0.86      2434
weighted avg       0.88      0.89      0.88      2434


CatBoost Classification Report:
              precision    recall  f1-score   support

           0       0.86      0.70      0.77       742
           1       0.88      0.95      0.91      1692

    accuracy                           0.87      2434
   macro avg       0.87      0.82      0.84      2434
weighted avg       0.87      0.87      0.87      2434


MLP Classification Report:
              precision    recall  f1-score   support

           0       0.00      0.00      0.00       742
           1       0.70      1.00      0.82      1692

    accuracy                           0.70      2434
   macro avg       0.35      0.50      0.41      2434
weighted avg       0.48      0.70      0.57      2434


Quadratic Discriminant Analysis Classification Report:
              precision    recall  f1-score   support

           0       0.33      0.97      0.50       742
           1       0.92      0.15      0.26      1692

    accuracy                           0.40      2434
   macro avg       0.63      0.56      0.38      2434
weighted avg       0.74      0.40      0.33      2434


Extra Trees Classification Report:
              precision    recall  f1-score   support

           0       0.84      0.81      0.82       742
           1       0.92      0.93      0.92      1692

    accuracy                           0.89      2434
   macro avg       0.88      0.87      0.87      2434
weighted avg       0.89      0.89      0.89      2434

Best parameters for Random Forest: {'max_depth': None, 'min_samples_split': 5, 'n_estimators': 200}
Best F1-score: 0.9293148321221679
Best parameters for Gradient Boosting: {'learning_rate': 0.3, 'max_depth': 7, 'n_estimators': 200}
Best F1-score: 0.9303527123419139
Best parameters for Naive Bayes: {}
Best F1-score: 0.8124441534677572
Best parameters for KNN: {'knn__metric': 'manhattan', 'knn__n_neighbors': 3, 'knn__weights': 'distance'}
Best F1-score: 0.9702072431513097
Best parameters for Logistic Regression: {'logreg__C': 0.01, 'logreg__solver': 'lbfgs'}
Best F1-score: 0.8176609276895921

Random Forest Classification Report:
              precision    recall  f1-score   support

           0       0.85      0.78      0.81       742
           1       0.91      0.94      0.92      1692

    accuracy                           0.89      2434
   macro avg       0.88      0.86      0.87      2434
weighted avg       0.89      0.89      0.89      2434


Gradient Boosting Classification Report:
              precision    recall  f1-score   support

           0       0.85      0.80      0.83       742
           1       0.91      0.94      0.93      1692

    accuracy                           0.90      2434
   macro avg       0.88      0.87      0.88      2434
weighted avg       0.90      0.90      0.90      2434


Naive Bayes Classification Report:
              precision    recall  f1-score   support

           0       1.00      0.00      0.00       742
           1       0.70      1.00      0.82      1692

    accuracy                           0.70      2434
   macro avg       0.85      0.50      0.41      2434
weighted avg       0.79      0.70      0.57      2434


KNN Classification Report:
              precision    recall  f1-score   support

           0       0.94      0.94      0.94       742
           1       0.97      0.97      0.97      1692

    accuracy                           0.96      2434
   macro avg       0.96      0.96      0.96      2434
weighted avg       0.96      0.96      0.96      2434


Logistic Regression Classification Report:
              precision    recall  f1-score   support

           0       0.65      0.23      0.34       742
           1       0.74      0.95      0.83      1692

    accuracy                           0.73      2434
   macro avg       0.69      0.59      0.58      2434
weighted avg       0.71      0.73      0.68      2434

[0]	train-auc:0.76568	eval-auc:0.75568
[1]	train-auc:0.78509	eval-auc:0.78014
[2]	train-auc:0.81380	eval-auc:0.80464
[3]	train-auc:0.82123	eval-auc:0.81147
[4]	train-auc:0.82870	eval-auc:0.81905
[5]	train-auc:0.83596	eval-auc:0.82512
[6]	train-auc:0.84242	eval-auc:0.82579
[7]	train-auc:0.84401	eval-auc:0.82637
[8]	train-auc:0.84393	eval-auc:0.82365
[9]	train-auc:0.84584	eval-auc:0.82448
[10]	train-auc:0.85439	eval-auc:0.83049
[11]	train-auc:0.85558	eval-auc:0.83124
[12]	train-auc:0.85985	eval-auc:0.83781
[13]	train-auc:0.85975	eval-auc:0.83772
[14]	train-auc:0.85961	eval-auc:0.83680
[15]	train-auc:0.86092	eval-auc:0.83931
[16]	train-auc:0.86503	eval-auc:0.84223
[17]	train-auc:0.86582	eval-auc:0.84304
[18]	train-auc:0.86807	eval-auc:0.84528
[19]	train-auc:0.86789	eval-auc:0.84410
[20]	train-auc:0.86771	eval-auc:0.84374
[21]	train-auc:0.87046	eval-auc:0.84721
[22]	train-auc:0.87127	eval-auc:0.84681
[23]	train-auc:0.87156	eval-auc:0.84762
[24]	train-auc:0.87286	eval-auc:0.84939
[25]	train-auc:0.87426	eval-auc:0.85127
[26]	train-auc:0.87440	eval-auc:0.85208
[27]	train-auc:0.87449	eval-auc:0.85242
[28]	train-auc:0.87467	eval-auc:0.85237
[29]	train-auc:0.87517	eval-auc:0.85280
[30]	train-auc:0.87525	eval-auc:0.85315
[31]	train-auc:0.87598	eval-auc:0.85362
[32]	train-auc:0.87708	eval-auc:0.85380
[33]	train-auc:0.87750	eval-auc:0.85367
[34]	train-auc:0.87705	eval-auc:0.85308
[35]	train-auc:0.87777	eval-auc:0.85385
[36]	train-auc:0.87878	eval-auc:0.85431
[37]	train-auc:0.87926	eval-auc:0.85441
[38]	train-auc:0.88014	eval-auc:0.85497
[39]	train-auc:0.88198	eval-auc:0.85692
[40]	train-auc:0.88236	eval-auc:0.85709
[41]	train-auc:0.88541	eval-auc:0.85928
[42]	train-auc:0.88639	eval-auc:0.85992
[43]	train-auc:0.88933	eval-auc:0.86267
[44]	train-auc:0.89108	eval-auc:0.86487
[45]	train-auc:0.89265	eval-auc:0.86630
[46]	train-auc:0.89272	eval-auc:0.86648
[47]	train-auc:0.89394	eval-auc:0.86831
[48]	train-auc:0.89400	eval-auc:0.86899
[49]	train-auc:0.89482	eval-auc:0.86904
[50]	train-auc:0.89536	eval-auc:0.86982
[51]	train-auc:0.89657	eval-auc:0.87147
[52]	train-auc:0.89772	eval-auc:0.87317
[53]	train-auc:0.89878	eval-auc:0.87395
[54]	train-auc:0.89910	eval-auc:0.87391
[55]	train-auc:0.90012	eval-auc:0.87415
[56]	train-auc:0.90029	eval-auc:0.87452
[57]	train-auc:0.90168	eval-auc:0.87603
[58]	train-auc:0.90177	eval-auc:0.87603
[59]	train-auc:0.90191	eval-auc:0.87605
[60]	train-auc:0.90195	eval-auc:0.87588
[61]	train-auc:0.90365	eval-auc:0.87781
[62]	train-auc:0.90453	eval-auc:0.87917
[63]	train-auc:0.90462	eval-auc:0.87910
[64]	train-auc:0.90582	eval-auc:0.88033
[65]	train-auc:0.90571	eval-auc:0.88039
[66]	train-auc:0.90583	eval-auc:0.88033
[67]	train-auc:0.90655	eval-auc:0.88126
[68]	train-auc:0.90841	eval-auc:0.88286
[69]	train-auc:0.90900	eval-auc:0.88307
[70]	train-auc:0.90936	eval-auc:0.88328
[71]	train-auc:0.91196	eval-auc:0.88701
[72]	train-auc:0.91258	eval-auc:0.88747
[73]	train-auc:0.91345	eval-auc:0.88829
[74]	train-auc:0.91344	eval-auc:0.88828
[75]	train-auc:0.91430	eval-auc:0.88971
[76]	train-auc:0.91470	eval-auc:0.89033
[77]	train-auc:0.91515	eval-auc:0.89072
[78]	train-auc:0.91604	eval-auc:0.89181
[79]	train-auc:0.91712	eval-auc:0.89247
[80]	train-auc:0.91727	eval-auc:0.89246
[81]	train-auc:0.91736	eval-auc:0.89262
[82]	train-auc:0.91885	eval-auc:0.89335
[83]	train-auc:0.91901	eval-auc:0.89367
[84]	train-auc:0.91967	eval-auc:0.89454
[85]	train-auc:0.92011	eval-auc:0.89482
[86]	train-auc:0.92048	eval-auc:0.89540
[87]	train-auc:0.92071	eval-auc:0.89547
[88]	train-auc:0.92058	eval-auc:0.89536
[89]	train-auc:0.92085	eval-auc:0.89558
[90]	train-auc:0.92119	eval-auc:0.89594
[91]	train-auc:0.92189	eval-auc:0.89624
[92]	train-auc:0.92221	eval-auc:0.89677
[93]	train-auc:0.92262	eval-auc:0.89739
[94]	train-auc:0.92280	eval-auc:0.89745
[95]	train-auc:0.92309	eval-auc:0.89750
[96]	train-auc:0.92342	eval-auc:0.89833
[97]	train-auc:0.92387	eval-auc:0.89863
[98]	train-auc:0.92415	eval-auc:0.89927
[99]	train-auc:0.92445	eval-auc:0.89960
[100]	train-auc:0.92495	eval-auc:0.90014
[101]	train-auc:0.92503	eval-auc:0.90051
[102]	train-auc:0.92513	eval-auc:0.90049
[103]	train-auc:0.92526	eval-auc:0.90049
[104]	train-auc:0.92556	eval-auc:0.90103
[105]	train-auc:0.92609	eval-auc:0.90137
[106]	train-auc:0.92615	eval-auc:0.90189
[107]	train-auc:0.92703	eval-auc:0.90309
[108]	train-auc:0.92714	eval-auc:0.90317
[109]	train-auc:0.92722	eval-auc:0.90326
[110]	train-auc:0.92729	eval-auc:0.90345
[111]	train-auc:0.92728	eval-auc:0.90352
[112]	train-auc:0.92736	eval-auc:0.90352
[113]	train-auc:0.92747	eval-auc:0.90363
[114]	train-auc:0.92779	eval-auc:0.90393
[115]	train-auc:0.92812	eval-auc:0.90401
[116]	train-auc:0.92823	eval-auc:0.90422
[117]	train-auc:0.92854	eval-auc:0.90479
[118]	train-auc:0.92866	eval-auc:0.90494
[119]	train-auc:0.92877	eval-auc:0.90499
[120]	train-auc:0.92887	eval-auc:0.90546
[121]	train-auc:0.92911	eval-auc:0.90595
[122]	train-auc:0.92931	eval-auc:0.90570
[123]	train-auc:0.92944	eval-auc:0.90570
[124]	train-auc:0.92953	eval-auc:0.90573
[125]	train-auc:0.92965	eval-auc:0.90585
[126]	train-auc:0.92991	eval-auc:0.90608
[127]	train-auc:0.93009	eval-auc:0.90620
[128]	train-auc:0.93044	eval-auc:0.90650
[129]	train-auc:0.93096	eval-auc:0.90689
[130]	train-auc:0.93149	eval-auc:0.90707
[131]	train-auc:0.93181	eval-auc:0.90765
[132]	train-auc:0.93176	eval-auc:0.90762
[133]	train-auc:0.93210	eval-auc:0.90799
[134]	train-auc:0.93254	eval-auc:0.90857
[135]	train-auc:0.93272	eval-auc:0.90862
[136]	train-auc:0.93315	eval-auc:0.90903
[137]	train-auc:0.93332	eval-auc:0.90915
[138]	train-auc:0.93359	eval-auc:0.90945
[139]	train-auc:0.93361	eval-auc:0.90933
[140]	train-auc:0.93374	eval-auc:0.90940
[141]	train-auc:0.93379	eval-auc:0.90956
[142]	train-auc:0.93415	eval-auc:0.90971
[143]	train-auc:0.93411	eval-auc:0.90972
[144]	train-auc:0.93430	eval-auc:0.90990
[145]	train-auc:0.93441	eval-auc:0.91024
[146]	train-auc:0.93460	eval-auc:0.91034
[147]	train-auc:0.93499	eval-auc:0.91066
[148]	train-auc:0.93501	eval-auc:0.91070
[149]	train-auc:0.93519	eval-auc:0.91096
[150]	train-auc:0.93522	eval-auc:0.91110
[151]	train-auc:0.93547	eval-auc:0.91107
[152]	train-auc:0.93566	eval-auc:0.91140
[153]	train-auc:0.93588	eval-auc:0.91196
[154]	train-auc:0.93620	eval-auc:0.91223
[155]	train-auc:0.93649	eval-auc:0.91246
[156]	train-auc:0.93680	eval-auc:0.91283
[157]	train-auc:0.93698	eval-auc:0.91308
[158]	train-auc:0.93710	eval-auc:0.91321
[159]	train-auc:0.93732	eval-auc:0.91319
[160]	train-auc:0.93751	eval-auc:0.91363
[161]	train-auc:0.93779	eval-auc:0.91400
[162]	train-auc:0.93781	eval-auc:0.91406
[163]	train-auc:0.93816	eval-auc:0.91435
[164]	train-auc:0.93814	eval-auc:0.91454
[165]	train-auc:0.93855	eval-auc:0.91460
[166]	train-auc:0.93869	eval-auc:0.91474
[167]	train-auc:0.93871	eval-auc:0.91463
[168]	train-auc:0.93875	eval-auc:0.91475
[169]	train-auc:0.93891	eval-auc:0.91469
[170]	train-auc:0.93918	eval-auc:0.91499
[171]	train-auc:0.93920	eval-auc:0.91498
[172]	train-auc:0.93925	eval-auc:0.91506
[173]	train-auc:0.93958	eval-auc:0.91559
[174]	train-auc:0.93975	eval-auc:0.91583
[175]	train-auc:0.94013	eval-auc:0.91643
[176]	train-auc:0.94031	eval-auc:0.91648
[177]	train-auc:0.94046	eval-auc:0.91661
[178]	train-auc:0.94069	eval-auc:0.91685
[179]	train-auc:0.94077	eval-auc:0.91694
[180]	train-auc:0.94079	eval-auc:0.91681
[181]	train-auc:0.94104	eval-auc:0.91743
[182]	train-auc:0.94123	eval-auc:0.91755
[183]	train-auc:0.94148	eval-auc:0.91758
[184]	train-auc:0.94162	eval-auc:0.91771
[185]	train-auc:0.94185	eval-auc:0.91790
[186]	train-auc:0.94257	eval-auc:0.91867
[187]	train-auc:0.94259	eval-auc:0.91876
[188]	train-auc:0.94256	eval-auc:0.91886
[189]	train-auc:0.94272	eval-auc:0.91908
[190]	train-auc:0.94276	eval-auc:0.91905
[191]	train-auc:0.94282	eval-auc:0.91907
[192]	train-auc:0.94316	eval-auc:0.91927
[193]	train-auc:0.94332	eval-auc:0.91919
[194]	train-auc:0.94362	eval-auc:0.91929
[195]	train-auc:0.94383	eval-auc:0.91968
[196]	train-auc:0.94381	eval-auc:0.91964
[197]	train-auc:0.94388	eval-auc:0.91969
[198]	train-auc:0.94392	eval-auc:0.91958
[199]	train-auc:0.94394	eval-auc:0.91959
Accuracy: 84.22%
ROC AUC: 0.92
              precision    recall  f1-score   support

           0       0.71      0.81      0.76       742
           1       0.91      0.86      0.88      1692

    accuracy                           0.84      2434
   macro avg       0.81      0.83      0.82      2434
weighted avg       0.85      0.84      0.84      2434

Epoch 1/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 1s 1ms/step - accuracy: 0.5643 - loss: 1782025.7500
Epoch 2/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5800 - loss: 1308241.8750
Epoch 3/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5599 - loss: 1015836.6875
Epoch 4/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5627 - loss: 774538.5625
Epoch 5/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5730 - loss: 1049485.7500
Epoch 6/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.5661 - loss: 575765.3125
Epoch 7/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5711 - loss: 1126287.0000
Epoch 8/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.5852 - loss: 982926.8125
Epoch 9/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5663 - loss: 1464671.3750
Epoch 10/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.5636 - loss: 1053796.8750
Test Accuracy: 0.6951519846916199
77/77 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step  
Classification Report for Deep Learning Model:
              precision    recall  f1-score   support

           0       0.00      0.00      0.00       742
           1       0.70      1.00      0.82      1692

    accuracy                           0.70      2434
   macro avg       0.35      0.50      0.41      2434
weighted avg       0.48      0.70      0.57      2434

Out[ ]:
<Sequential name=sequential, built=True>

Label¶

In [ ]:
# Split data
X_train_Le, X_test_Le, y_train_Le, y_test_Le = PreprocessingTrainTestSplit.split_data(result_df_Le, "corrisponde_ad_attacco")

# Initial model training and evaluation
InitialTraining.train_and_evaluate_initial_models(X_train_Le, y_train_Le, X_test_Le, y_test_Le)

# Hyperparameter tuning
best_models_Le = HyperparameterTuning.tune_hyperparameters(X_train_Le, y_train_Le)

# Evaluate best models on test set
evaluator_Le = ModelEvaluator(best_models_Le)
evaluation_results_Le = evaluator_Le.evaluate_models(X_test_Le, y_test_Le)

# Train XGBoost model
AdvancedModels.train_xgboost(X_train_Le, y_train_Le, X_test_Le, y_test_Le)

# Train deep learning model
DeepLearningModel.train_deep_learning_model(X_train_Le, y_train_Le, X_test_Le, y_test_Le)
Decision Tree Classification Report:
              precision    recall  f1-score   support

           0       0.83      0.82      0.83       742
           1       0.92      0.93      0.92      1692

    accuracy                           0.89      2434
   macro avg       0.88      0.88      0.88      2434
weighted avg       0.89      0.89      0.89      2434


AdaBoost Classification Report:
              precision    recall  f1-score   support

           0       0.74      0.29      0.42       742
           1       0.75      0.96      0.84      1692

    accuracy                           0.75      2434
   macro avg       0.75      0.62      0.63      2434
weighted avg       0.75      0.75      0.71      2434


XGBoost Classification Report:
              precision    recall  f1-score   support

           0       0.86      0.77      0.81       742
           1       0.90      0.95      0.92      1692

    accuracy                           0.89      2434
   macro avg       0.88      0.86      0.87      2434
weighted avg       0.89      0.89      0.89      2434


CatBoost Classification Report:
              precision    recall  f1-score   support

           0       0.86      0.71      0.78       742
           1       0.88      0.95      0.91      1692

    accuracy                           0.88      2434
   macro avg       0.87      0.83      0.85      2434
weighted avg       0.88      0.88      0.87      2434


MLP Classification Report:
              precision    recall  f1-score   support

           0       0.00      0.00      0.00       742
           1       0.70      1.00      0.82      1692

    accuracy                           0.70      2434
   macro avg       0.35      0.50      0.41      2434
weighted avg       0.48      0.70      0.57      2434


Quadratic Discriminant Analysis Classification Report:
              precision    recall  f1-score   support

           0       0.49      0.19      0.28       742
           1       0.72      0.91      0.81      1692

    accuracy                           0.69      2434
   macro avg       0.61      0.55      0.54      2434
weighted avg       0.65      0.69      0.64      2434


Extra Trees Classification Report:
              precision    recall  f1-score   support

           0       0.84      0.81      0.82       742
           1       0.92      0.93      0.92      1692

    accuracy                           0.89      2434
   macro avg       0.88      0.87      0.87      2434
weighted avg       0.89      0.89      0.89      2434

Best parameters for Random Forest: {'max_depth': 20, 'min_samples_split': 2, 'n_estimators': 300}
Best F1-score: 0.9328604085098989
Best parameters for Gradient Boosting: {'learning_rate': 0.3, 'max_depth': 7, 'n_estimators': 300}
Best F1-score: 0.9324297787118544
Best parameters for Naive Bayes: {}
Best F1-score: 0.8117828645683514
Best parameters for KNN: {'knn__metric': 'manhattan', 'knn__n_neighbors': 9, 'knn__weights': 'distance'}
Best F1-score: 0.9771909225408988
Best parameters for Logistic Regression: {'logreg__C': 0.01, 'logreg__solver': 'liblinear'}
Best F1-score: 0.8097576181622838

Random Forest Classification Report:
              precision    recall  f1-score   support

           0       0.87      0.79      0.83       742
           1       0.91      0.95      0.93      1692

    accuracy                           0.90      2434
   macro avg       0.89      0.87      0.88      2434
weighted avg       0.90      0.90      0.90      2434


Gradient Boosting Classification Report:
              precision    recall  f1-score   support

           0       0.86      0.81      0.83       742
           1       0.92      0.94      0.93      1692

    accuracy                           0.90      2434
   macro avg       0.89      0.88      0.88      2434
weighted avg       0.90      0.90      0.90      2434


Naive Bayes Classification Report:
              precision    recall  f1-score   support

           0       0.00      0.00      0.00       742
           1       0.70      1.00      0.82      1692

    accuracy                           0.70      2434
   macro avg       0.35      0.50      0.41      2434
weighted avg       0.48      0.70      0.57      2434


KNN Classification Report:
              precision    recall  f1-score   support

           0       0.95      0.95      0.95       742
           1       0.98      0.98      0.98      1692

    accuracy                           0.97      2434
   macro avg       0.96      0.96      0.96      2434
weighted avg       0.97      0.97      0.97      2434


Logistic Regression Classification Report:
              precision    recall  f1-score   support

           0       0.49      0.02      0.05       742
           1       0.70      0.99      0.82      1692

    accuracy                           0.69      2434
   macro avg       0.59      0.51      0.43      2434
weighted avg       0.63      0.69      0.58      2434

[0]	train-auc:0.79334	eval-auc:0.76224
[1]	train-auc:0.81766	eval-auc:0.77658
[2]	train-auc:0.84054	eval-auc:0.81116
[3]	train-auc:0.84337	eval-auc:0.81958
[4]	train-auc:0.85193	eval-auc:0.82875
[5]	train-auc:0.85569	eval-auc:0.83242
[6]	train-auc:0.85806	eval-auc:0.83798
[7]	train-auc:0.85961	eval-auc:0.83771
[8]	train-auc:0.86074	eval-auc:0.84046
[9]	train-auc:0.86104	eval-auc:0.84106
[10]	train-auc:0.86334	eval-auc:0.84423
[11]	train-auc:0.86248	eval-auc:0.84302
[12]	train-auc:0.86743	eval-auc:0.84778
[13]	train-auc:0.86726	eval-auc:0.84784
[14]	train-auc:0.86811	eval-auc:0.84833
[15]	train-auc:0.87042	eval-auc:0.84889
[16]	train-auc:0.87092	eval-auc:0.84964
[17]	train-auc:0.87668	eval-auc:0.85663
[18]	train-auc:0.87711	eval-auc:0.85868
[19]	train-auc:0.88037	eval-auc:0.86024
[20]	train-auc:0.88371	eval-auc:0.86271
[21]	train-auc:0.88373	eval-auc:0.86344
[22]	train-auc:0.88479	eval-auc:0.86438
[23]	train-auc:0.88479	eval-auc:0.86406
[24]	train-auc:0.88528	eval-auc:0.86467
[25]	train-auc:0.88595	eval-auc:0.86517
[26]	train-auc:0.88877	eval-auc:0.86710
[27]	train-auc:0.89219	eval-auc:0.87088
[28]	train-auc:0.89245	eval-auc:0.87072
[29]	train-auc:0.89273	eval-auc:0.87067
[30]	train-auc:0.89557	eval-auc:0.87294
[31]	train-auc:0.89553	eval-auc:0.87270
[32]	train-auc:0.89798	eval-auc:0.87413
[33]	train-auc:0.89797	eval-auc:0.87401
[34]	train-auc:0.89818	eval-auc:0.87379
[35]	train-auc:0.89845	eval-auc:0.87426
[36]	train-auc:0.89928	eval-auc:0.87521
[37]	train-auc:0.89979	eval-auc:0.87513
[38]	train-auc:0.90022	eval-auc:0.87579
[39]	train-auc:0.90066	eval-auc:0.87637
[40]	train-auc:0.90080	eval-auc:0.87641
[41]	train-auc:0.90188	eval-auc:0.87830
[42]	train-auc:0.90250	eval-auc:0.87869
[43]	train-auc:0.90296	eval-auc:0.87945
[44]	train-auc:0.90451	eval-auc:0.88107
[45]	train-auc:0.90589	eval-auc:0.88192
[46]	train-auc:0.90624	eval-auc:0.88193
[47]	train-auc:0.90653	eval-auc:0.88236
[48]	train-auc:0.90672	eval-auc:0.88223
[49]	train-auc:0.90815	eval-auc:0.88298
[50]	train-auc:0.90867	eval-auc:0.88361
[51]	train-auc:0.91014	eval-auc:0.88527
[52]	train-auc:0.91112	eval-auc:0.88603
[53]	train-auc:0.91142	eval-auc:0.88596
[54]	train-auc:0.91238	eval-auc:0.88688
[55]	train-auc:0.91404	eval-auc:0.88883
[56]	train-auc:0.91422	eval-auc:0.88939
[57]	train-auc:0.91615	eval-auc:0.89180
[58]	train-auc:0.91614	eval-auc:0.89193
[59]	train-auc:0.91631	eval-auc:0.89192
[60]	train-auc:0.91667	eval-auc:0.89244
[61]	train-auc:0.91721	eval-auc:0.89352
[62]	train-auc:0.91719	eval-auc:0.89298
[63]	train-auc:0.91771	eval-auc:0.89370
[64]	train-auc:0.91808	eval-auc:0.89468
[65]	train-auc:0.91917	eval-auc:0.89548
[66]	train-auc:0.91933	eval-auc:0.89552
[67]	train-auc:0.91923	eval-auc:0.89458
[68]	train-auc:0.91991	eval-auc:0.89583
[69]	train-auc:0.92107	eval-auc:0.89735
[70]	train-auc:0.92133	eval-auc:0.89744
[71]	train-auc:0.92176	eval-auc:0.89795
[72]	train-auc:0.92252	eval-auc:0.89754
[73]	train-auc:0.92309	eval-auc:0.89765
[74]	train-auc:0.92366	eval-auc:0.89832
[75]	train-auc:0.92440	eval-auc:0.89913
[76]	train-auc:0.92480	eval-auc:0.89989
[77]	train-auc:0.92596	eval-auc:0.90128
[78]	train-auc:0.92635	eval-auc:0.90128
[79]	train-auc:0.92658	eval-auc:0.90152
[80]	train-auc:0.92752	eval-auc:0.90224
[81]	train-auc:0.92787	eval-auc:0.90313
[82]	train-auc:0.92836	eval-auc:0.90381
[83]	train-auc:0.92877	eval-auc:0.90440
[84]	train-auc:0.92912	eval-auc:0.90477
[85]	train-auc:0.92965	eval-auc:0.90536
[86]	train-auc:0.93076	eval-auc:0.90641
[87]	train-auc:0.93093	eval-auc:0.90658
[88]	train-auc:0.93102	eval-auc:0.90709
[89]	train-auc:0.93155	eval-auc:0.90742
[90]	train-auc:0.93210	eval-auc:0.90740
[91]	train-auc:0.93216	eval-auc:0.90736
[92]	train-auc:0.93249	eval-auc:0.90757
[93]	train-auc:0.93243	eval-auc:0.90747
[94]	train-auc:0.93271	eval-auc:0.90764
[95]	train-auc:0.93319	eval-auc:0.90835
[96]	train-auc:0.93395	eval-auc:0.90951
[97]	train-auc:0.93431	eval-auc:0.90990
[98]	train-auc:0.93459	eval-auc:0.91070
[99]	train-auc:0.93522	eval-auc:0.91101
[100]	train-auc:0.93613	eval-auc:0.91178
[101]	train-auc:0.93642	eval-auc:0.91206
[102]	train-auc:0.93683	eval-auc:0.91230
[103]	train-auc:0.93709	eval-auc:0.91254
[104]	train-auc:0.93798	eval-auc:0.91300
[105]	train-auc:0.93803	eval-auc:0.91307
[106]	train-auc:0.93834	eval-auc:0.91325
[107]	train-auc:0.93872	eval-auc:0.91383
[108]	train-auc:0.93893	eval-auc:0.91400
[109]	train-auc:0.93906	eval-auc:0.91434
[110]	train-auc:0.93915	eval-auc:0.91453
[111]	train-auc:0.93935	eval-auc:0.91479
[112]	train-auc:0.93943	eval-auc:0.91494
[113]	train-auc:0.93953	eval-auc:0.91505
[114]	train-auc:0.93975	eval-auc:0.91517
[115]	train-auc:0.93991	eval-auc:0.91551
[116]	train-auc:0.94008	eval-auc:0.91554
[117]	train-auc:0.94037	eval-auc:0.91589
[118]	train-auc:0.94067	eval-auc:0.91604
[119]	train-auc:0.94101	eval-auc:0.91609
[120]	train-auc:0.94134	eval-auc:0.91641
[121]	train-auc:0.94142	eval-auc:0.91639
[122]	train-auc:0.94153	eval-auc:0.91639
[123]	train-auc:0.94160	eval-auc:0.91637
[124]	train-auc:0.94180	eval-auc:0.91667
[125]	train-auc:0.94189	eval-auc:0.91678
[126]	train-auc:0.94213	eval-auc:0.91702
[127]	train-auc:0.94225	eval-auc:0.91730
[128]	train-auc:0.94255	eval-auc:0.91753
[129]	train-auc:0.94312	eval-auc:0.91797
[130]	train-auc:0.94340	eval-auc:0.91831
[131]	train-auc:0.94359	eval-auc:0.91820
[132]	train-auc:0.94417	eval-auc:0.91884
[133]	train-auc:0.94461	eval-auc:0.91924
[134]	train-auc:0.94459	eval-auc:0.91917
[135]	train-auc:0.94489	eval-auc:0.91938
[136]	train-auc:0.94523	eval-auc:0.92006
[137]	train-auc:0.94528	eval-auc:0.91989
[138]	train-auc:0.94540	eval-auc:0.91992
[139]	train-auc:0.94563	eval-auc:0.92026
[140]	train-auc:0.94597	eval-auc:0.92047
[141]	train-auc:0.94608	eval-auc:0.92060
[142]	train-auc:0.94625	eval-auc:0.92048
[143]	train-auc:0.94644	eval-auc:0.92079
[144]	train-auc:0.94675	eval-auc:0.92112
[145]	train-auc:0.94695	eval-auc:0.92113
[146]	train-auc:0.94720	eval-auc:0.92123
[147]	train-auc:0.94731	eval-auc:0.92143
[148]	train-auc:0.94732	eval-auc:0.92154
[149]	train-auc:0.94743	eval-auc:0.92163
[150]	train-auc:0.94770	eval-auc:0.92170
[151]	train-auc:0.94790	eval-auc:0.92184
[152]	train-auc:0.94822	eval-auc:0.92199
[153]	train-auc:0.94831	eval-auc:0.92204
[154]	train-auc:0.94838	eval-auc:0.92190
[155]	train-auc:0.94873	eval-auc:0.92237
[156]	train-auc:0.94877	eval-auc:0.92255
[157]	train-auc:0.94908	eval-auc:0.92252
[158]	train-auc:0.94954	eval-auc:0.92304
[159]	train-auc:0.94970	eval-auc:0.92331
[160]	train-auc:0.94983	eval-auc:0.92343
[161]	train-auc:0.94995	eval-auc:0.92364
[162]	train-auc:0.95004	eval-auc:0.92371
[163]	train-auc:0.95040	eval-auc:0.92383
[164]	train-auc:0.95075	eval-auc:0.92393
[165]	train-auc:0.95106	eval-auc:0.92416
[166]	train-auc:0.95104	eval-auc:0.92407
[167]	train-auc:0.95124	eval-auc:0.92425
[168]	train-auc:0.95132	eval-auc:0.92417
[169]	train-auc:0.95149	eval-auc:0.92433
[170]	train-auc:0.95184	eval-auc:0.92472
[171]	train-auc:0.95214	eval-auc:0.92510
[172]	train-auc:0.95246	eval-auc:0.92538
[173]	train-auc:0.95249	eval-auc:0.92552
[174]	train-auc:0.95264	eval-auc:0.92563
[175]	train-auc:0.95280	eval-auc:0.92571
[176]	train-auc:0.95291	eval-auc:0.92595
[177]	train-auc:0.95298	eval-auc:0.92620
[178]	train-auc:0.95328	eval-auc:0.92642
[179]	train-auc:0.95334	eval-auc:0.92661
[180]	train-auc:0.95344	eval-auc:0.92680
[181]	train-auc:0.95357	eval-auc:0.92717
[182]	train-auc:0.95378	eval-auc:0.92726
[183]	train-auc:0.95396	eval-auc:0.92724
[184]	train-auc:0.95418	eval-auc:0.92745
[185]	train-auc:0.95438	eval-auc:0.92752
[186]	train-auc:0.95458	eval-auc:0.92769
[187]	train-auc:0.95457	eval-auc:0.92783
[188]	train-auc:0.95457	eval-auc:0.92784
[189]	train-auc:0.95467	eval-auc:0.92797
[190]	train-auc:0.95478	eval-auc:0.92784
[191]	train-auc:0.95491	eval-auc:0.92803
[192]	train-auc:0.95515	eval-auc:0.92843
[193]	train-auc:0.95516	eval-auc:0.92822
[194]	train-auc:0.95535	eval-auc:0.92837
[195]	train-auc:0.95552	eval-auc:0.92836
[196]	train-auc:0.95565	eval-auc:0.92862
[197]	train-auc:0.95562	eval-auc:0.92855
[198]	train-auc:0.95583	eval-auc:0.92859
[199]	train-auc:0.95584	eval-auc:0.92857
Accuracy: 85.95%
ROC AUC: 0.93
              precision    recall  f1-score   support

           0       0.74      0.83      0.78       742
           1       0.92      0.87      0.90      1692

    accuracy                           0.86      2434
   macro avg       0.83      0.85      0.84      2434
weighted avg       0.87      0.86      0.86      2434

Epoch 1/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 2s 1ms/step - accuracy: 0.5767 - loss: 2932992.5000
Epoch 2/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5477 - loss: 2713778.0000
Epoch 3/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5695 - loss: 2416932.2500
Epoch 4/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5606 - loss: 1846083.6250
Epoch 5/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5566 - loss: 2680881.0000
Epoch 6/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5673 - loss: 2293997.5000
Epoch 7/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.5547 - loss: 1850834.0000
Epoch 8/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5722 - loss: 1339577.6250
Epoch 9/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5605 - loss: 2254119.0000
Epoch 10/10
229/229 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5698 - loss: 2419570.0000
Test Accuracy: 0.30484798550605774
77/77 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step  
Classification Report for Deep Learning Model:
              precision    recall  f1-score   support

           0       0.30      1.00      0.47       742
           1       0.00      0.00      0.00      1692

    accuracy                           0.30      2434
   macro avg       0.15      0.50      0.23      2434
weighted avg       0.09      0.30      0.14      2434

Out[ ]:
<Sequential name=sequential_1, built=True>
In [ ]:
evaluator_OH.print_best_model('OneHot Encoder')
evaluator_Le.print_best_model('Label Encoder')
Dopo la codifica con OneHot Encoder il modello migliore è stato KNN con lo score di 0.9559

Dopo la codifica con Label Encoder il modello migliore è stato KNN con lo score di 0.9612